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ABSTRACT 


Two  proposal  competitions  for 
government  R  S  D  contract,  involving 
9  and  12  firms  respectively,  are 
examined.   Ten  variables  characterizing 
the  competing  laboratories  and  their 
research  procedures  are  measured  and 
correlated  with  the  technical  ranking 
of  the  proposals  as  determined  by  the 
responsiable  government  laboratory 
without  respect  to  the  bid  price.   In 
both  sets  of  proposals  the  technical 
quality  is  uncorrelated  with  the  size 
of  the  proposal  preparation  team  but  is 
positively  related  to  the  size  of  the 
laboratory  and  especially  to  the  total 
number  of  engineers  and  scientists  in 
the  laboratory.   Effort  (total  man- 
hours)  is  positively  correlated  (,31, 
,33)  with  technical  rank.   Time  spent 
consulting  specialists  in  the  labora- 
tory is  related  in  both  cases  (,37,  ,55), 
but  time  spent  in  analytic  design  is 
negctively  related  to  technical  quality 
on  the  smaller  job  and  positively  on  the 
larger  (-,29,  ,38).   Time  spent  in 
literature  search  or  in  consulting  people 
outside  the  laboratory  shows  no  positive 
relation. 
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Research  by  its  very  nature  is  an  uncertain  activity., 
Some  projects  fulfill  or  exceed  expectations  for  their 
success;  others  fail  in  varying  degree.   There  is  at  present 
little  systematic  knowledge  about  the  factors  leading  to 
success  or  failure.   Many  considerations  can  affect  the 
outcome  of  a  research  and  development  project.   Some  of 
these  are  fairly  obvious;  for  example,  the  technical  pro- 
ficiency (quality)  of  the  researchers,  the  technical  manpower 
(quantity)  employed  on  the  task,  the  length  of  time  devoted 
to  the  job.   Even  among  the  more  obvious  factors,  however, 
little  is  known  about  their  relative  importance. 

The  investigation  of  factors  in  research  performance 
in  real  life  situations  on  major  problems  is  made  difficult 
by  the  fact  that  a  particular  problem  is  worked  on  only  once, 
Amost  by  definition  a  research  problem  is  unique;  if  it  has 
been  solved  before,  it  is  no  longer  a  research  problem. 
Replication  thus  is  infrequent. 

In  addition  to  a  few  instances  of  parallel  effort  in 
government  contracting,  replication  can  be  found  in  the  pro- 
posal competition  for  government  contracts.   There  are  two 
additional  reasons  for  selecting  this  phase  for  investigation. 
First,  it  is  the  stage  in  which  major  decisions  are  made 
regarding  the  overall  approach  to  the  problems  involved.   The 
preliminary  design  of  a  system  is  accomplished  at  this  time 
and  any  changes  subsequently  made  in  the  design  are  usually 
minor  by  comparison.   Secondly,  this  portion  of  the  R  £  D 
effort  is  normally  performed  by  a  number  of  competing  firms 
working  concurrently  and  often  selecting  different  approaches 
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to  the  solution  of  the  same  problem.   The  technical  eval- 
uation of  the  resulting  proposals  by  the  government  labor- 
atory provides  a  rating  against  which  a  number  of  factors 
characterizing  the  different  approaches  taken  by  competing 
firms  may  be  correlated. 

In  contracting  for  research  and  development,  a  govern- 
ment agency  usually  solicits  proposals  from  a  number  of  firms 
having  previous  experience  or  interest  in  the  relevant  field 
of  endeavor.   The  technical  staff  of  the  agency  draws  up  a 
work  statement  describing  the  mission  the  system  is  to  perform 
and  setting  forth  certain  criteria  to  which  the  design  must 
conform.   This  work  statement  is  incorporated  in  a  formal 
Request  for  Proposal  which  is  sent  to  the  several  firms.   The 
preparation  of  proposals  and  preliminary  designs  may  be 
described  as  a  problem-solving  process  carried  out  under  con- 
ditions of  competition  and  high  reward,  and  with  cost  penalties 
for  wasted  or  ineffective  effort. 

RESEARCH  METHODS 

Two  contracts  were  selected  for  study  from  the  files 
of  the  Air  Force  Electronic  Systems  Division.   The  Air  Force 
Cambridge  Research  Laboratory  (CRD  was  the  agency  responsible 
for  technical  evaluation  of  the  proposals  for  these  contracts. 
The  contract  files  contain  a  copy  of  the  work  statement  for 
the  task,  copies  of  all  proposals  submitted,  and  a  report  of 
the  technical  evaluation  performed  by  CRL.   The  latter  document 
indicates,  with  substantiating  reasons,  which  proposals  were 
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acceptable  and  which  were  unacceptable,  and  lists  them  in 
order  of  their  technical  quality.    This  evaluation  is  made  by 
technical  specialists  who  are  deliberately  kept  in  ignorance  of 
the  bid  cost  of  the  several  proposals.   The  final  contract 
awards  are  made  by  others  who  consider  both  technical  quality 
and  cost.   The  ordinal  scale  of  technical  quality  thus  provided 
is  the  dependent  variable  in  our  analysis,  and  several  indepen- 
dent variables  are  measured  to  determine  their  relation  to 
technical  quality. 

The  work  statements,  proposals,  and  CRL  reports  relating 
to  each  of  the  contracts  were  studied  carefully  to  obtain  a 
general  understanding  of  CRL's  desires  and  the  situation  con- 
fronting the  firms  preparing  the  proposals.   The  two  problems 
faced  by  the  competing  firms  may  be  stated  as  follows: 

Problem  A:   To  design  a  portable  meteorological  set  to 
measure  wind  velocity,  air  temperature,  relative  humidity,  and 
atmospheric  pressure.   It  was  to  be  capable  of  withstanding  a 
parachute  drop  from  an  airplane,  or  a  ten-foot  free  fall.   The 
set  was  to  be  operated  by  one  man  and  was  to  be  powered  by 
self-contained  batteries. 

Problem  B:  To  provide  an  experim.ental  microwave  antenna 
system  which  would  have  a  design  frequency  of  35,000  megacycles 
and  meet  extremely  severe  requirements  in  antenna  gain 


•^The  evaluators  of  Problem  B  were  reluctant,  to  make  a  com- 
plete ordinal  ranking  of  the  solutions  proposed  by  the  twelve 
firms  in  the  sample.   Seven  were  evaluated  as  "acceptable" 
and  ranked  in  order  of  technical  quality,  and  of  the  five 
remaining  "unacceptable"  solutions,  the  ranking  of  only  the 
last-place  firm  is  known.   This  results  in  a  four-way  tie  for 
ninth  place  in  the  ranking. 
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(68  db),  sidelobe  suppression  (25  db  within  a  1  degree  cone, 
55  db  elsewhere),  noise  temperature  (20  degrees  Kelvin), 
minimum  scan  rate  (no  greater  than  0.0005  degree  sec."  ),  and 
pointing  accuracy  (+_  0.01  degree). 

Information  concerning  the  process  of  proposal  pre- 
paration was  secured  from  all  the  firms  that  had  submitted 
proposals.   A  set  of  interview  questions  was  designed  for 
each  of  the  problems.   The  questions  were  based  upon  know- 
ledge obtained  from  the  work  statements,  proposals,  and 
interviews  with  technical  evaluators .   They  were  tailored 
to  the  individual  contracts,  but  attempted  to  elicit  compar- 
able information  on  the  ten  factors  listed  in  Table  XI.   In 
each  case,  one  or  more  members  of  the  proposal  team,  usually 
including  the  proposal  manager,  were  sought  out  for  inter- 
view.  An  interview  usually  lasted  one  hour  but  in  some  cases 
the  subjects  were  so  interested  in  the  topic  that  their 
interviews  continued  several  hours. 

Because  of  geographic  and  time  constraints,  not  all  of 
the  competing  laboratories  could  be  reached  for  detailed 
interview.   Those  which  could  not  be  reached  were  sent  a 
questionnaire  dealing  with  the  same  factors.   Of  thirteen 
laboratories  that  received  questionnaires,  ten  responded. 
The  size  of  the  sample  for  Problem  A  is  nine  firms.   Five  of 
these  were  covered  by  direct  interview  and  four  completed  the 
questionnaires  mailed  to  them.   For  Problem  B  the  sample  size 
is  twelve,  six  by  interview  and  six  by  questionnaire. 

Of  the  many  factors  that  might  influence  the  technical 
quality  of  proposals  for  government  contract,  ten  were  selected, 


measured,  and  tested  (see  l|st  in  Table  XI).   A  nonparametric 
measure  of  correlation  (Kendall  tau  coefficient)  was  chosen 
since  true  interval  scaling  is  not  available  for  the  technical 
evaluation  of  the  solutions. 

FACTORS  INFLUENCING  TECHNICAL  QUALITY 

1 .   Effect  of  Cost  on  Other  Correlations 
The  term  "cost"  refers  to  the  price  quoted  in  the  pro- 
posal, the  cost  for  which  the  firm  proposes  to  perform  the 
job.   One  might  imagine  that  a  firm  proposing  one  of  the  cost- 
lier solutions  to  the  problem  would  perhaps  have  a  relatively 
more  sophisticated  solution  and  therefore  receive  a  higher 
ranking  in  the  technical  evaluation.   Conversely,  a  firm  sus- 
pecting that  the  government  had  a  rather  limited  budget  for 
the  project  might  pare  down  its  technical  proposal  to  cut 
costs  and  thereby  suffer  in  the  technical  evaluation.   For 
these  reasons  it  is  conceivable  that  cost  might  have  a  major 
effect  upon  other  correlations.   This  possibility  was  checked 
by  partialling  cost  out  of  all  correlations  and  examining  the 
effect.   Since  the  change  was  in  all  cases  negligible,  the 
effect  of  cost  upon  other  correlations  can  be  ignored  in  the 
ensuing  analysis. 

Table  I  shows  the  relation  to  technical  quality  of  the 
costs  projected  by  the  competing  firms  for  Problems  A  and  B. 
(At  this  writing,  contract  A  has  been  completed  at  a  final 
cost  of  $18,59  3.   Contract  B  is  about  20  per  cent  completed 


TABLE  I 
RELATION  BETWEEN  COST  AND  TECHNICAL  QUALITY 


No.  of    Median      Range      Corre- 
Problem   labs      Cost       of  costs   lation   p 


A 

9 

20 

7-98 

.11 

.38 

B 

12 

795 

327-1,114 

.29 

.09 

Cost  figures  are  in  thousands  of  dollars 

p  =  Probability  of  occurrence  of  a  correlation  this 

high  if  there  were  no  relation  between  the 

variables . 


with  an  anticipated  cost  overrun  of  36  per  cent.) 

2 .   Level  of  Effort  Expended  on  the  Problem 
Level  of  effort  is  measured  in  terms  of  the  total  number 
of  technical  man-hours  expended  in  proposal  preparation.   The 
results  for  the  two  problems  (Table  II)  agree  remarkably  well 
in  spite  of  the  many  differences  between  the  problems  and  among 
the  problem-solving  groups.   Technical  quality  appears  to  be 
moderately  related  to  level  of  effort  expended.   There  may  be 
two  opposing  factors  influencing  this  relation,  and  resulting 
in  a  U-shaped  curve  for  level  of  effort.   Among  the  more  compe- 
tent firms  level  of  effort  may  have  a  strong  positive  relation 

TABLE  II 
RELATION  OF  LEVEL  OF  EFFORT  TO  TECHNICAL  QUALITY 


No,  of  Median  Range  of  Corre- 

Problem    labs  effort    effort  lation  p 

A        9        70      14-200     ,31  .15 

B       10       565  100-1440    ,33  .07 
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to  technical  quality  while  among  less  competent  firms  some 
may  attempt  to  substitute  energy  for  competence  and  thus 
attenuate  the  relation.   Table  II-A  shows  that  for  proposals 

TABLE  II-A 

RELATION  OF  LEVEL  OF  EFFORT  TO  TECHNICAL  QUALITY  OF  PROPOSALS 
DIVIDED  INTO  UPPER  AND  LOWER  HALVES  OF  THE  RANKING 


Technical  Ranking  No,  of   Corre- 
Problem     of  Proposal      labs    lation   p 


A£B 


upper 

half 

5 

.60 

,12 

lower 

half 

4 

-,67 

,17 

upper 

half 

5 

,40 

.24 

lower 

half 

6 

,07 

,50 

upper 

half 

10 

,57 

.02 

lower 

half 

10 

-,1U 

,33 

in  the  upper  half  of  the  ranking,  there  is  a  strong  positive 
relation  between  level  of  effort  and  technical  quality;  for  those 
in  the  lower  half  there  is  an  inverse  relation  or  none  at  all, 

3 .   Educational  Level  of  Proposal  Team 

Lacking  any  good  measure  of  technical  competence  of  the 
proposal  team  members,  we  decided  to  rank  the  mean  level  of 
educational  attainment.   Researchers  were  assigned  scores 
according  to  the  following  scale: 

0   Non-technical  degree  (e.g,  business  administration) 

0  No  college  degree  but  engineering  job  classification 

1  No  college  degree  but  engineering  job  classification 

plus  five  years  experience 

2  BoS,  engineering  or  science 

3  MoS,  engineering  or  science 

4  Pho  D,  engineering  or  science 


A  score  was  obtained  for  each  proposal  team  by  averaging  the 
scores  of  the  key  members » 

TABLE  III 

RELATION  OF  EDUCATIONAL  LEVEL  OF  PERSONS  PREPARING 
PROPOSAL  TO  TECHNICAL  QUALITY 


No.  of   Median   Range  of   Corre- 
Problem    labs    level     levels    lation   p 


A 

9 

2.0 

0-2.5 

.04 

.50 

B 

12 

lo58 

0-2.5 

.10 

,32 

The  low  correlations  are  due  in  part  to  the  large  number  of 
tied  scores  resulting  in  a  narrow  dispersion. 

4.   Time  Spent  in  Analytic  Design 

Each  firm  was  asked,  "How  many  man-hours  were  spent  in 
analytic  design  of  this  system  and  in  comparison  of  the  various 
alternatives  available  in  the  several  components  of  the  system?' 

TABLE  IV 
RELATION  OF  TIME  SPENT  IN  ANALYTIC  DESIGN  TO  TECHNICAL  QUALITY 


No  0  of   Median   Range     Corre- 
Problem    labs     time    of  time   lation   p 


9 

10.5 

6-40 

-.29 

,18 

12 

315 

50-1300 

f 
.38 

,04 

Table  IV  shows  that  time  spent  in  analytic  design  appears  to 
be  a  factor  in  technical  quality,  for  Problem  B  at  least.   The 
inverse  relation  on  Problem  A  is  difficult  to  explain,  but  may 


be  due  in  part  to  the  nature  of  the  problem.   From  an  analytic 
viewpoint,  this  was  a  much  simpler  problem  than  the  second. 
It  involved  selecting  the  proper  instruments  to  meet  the 
environmental  and  accuracy  specifications,  and  bringing  them 
together  in  a  single-package  design.   In  most  cases  instru- 
ments were  already  available  which  would  perform  the  job  and 
the  only  analytic  design  necessary  was  that  required  to  adapt 
an  available  instrument  to  the  specified  mission.   The  nega- 
tive correlation  might  indicate  that  more  homework  was 
required  on  the  part  of  the  less  experienced  firms  to  arrive 
at  a  (less  satisfactory)  solution.  On  the  second  problem, 
analytic  design  was  a  more  important  component  of  the  solution 
process,  hence  the  higher  correlation.   Mean  per  cent  of  total 
proposal  effort  spent  in  analytic  design  on  Problem  A  was 
25.3  per  cent,  on  Problem  B,  47.8  per  cent. 

Yet  another  factor  may  have  contributed  to  the  negative 
correlation  on  Problem  A,   The  laboratory  ranked  highest  had 
just  completed  a  rather  extensive  design  effort  on  one  of  the 
components  for  another  application.   If  the  time  spent  in 
analytic  design  for  this  other  application  is  counted  in  their 
total,  the  correlation  becomes  .03.   This  still  indicates  that 
analytic  design  effort  was  relatively  unimportant  in  the  solu- 
tion of  this  problem  but  perhaps  explains  the  negative  value 
obtained  on  the  original  correlation. 

5 .   Time  Spent  in  Literature  Search 

No  correlation  is  found  between  time  spent  in  literature 
search  and  technical  quality  for  either  problem  (Table  V). 
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These  results  would  lead  one  to  conclude  that  literature  search 
is  not  a  good  source  of  information  for  solving  problems 

TABLE  V 
RELATION  OF  TIME  SPENT  IN  LITERATURE  SEARCH  TO  TECHNICAL  QUALITY 

No.  of  Median  Range  of   Corre- 
Problem   labs    time     time    lation   p 

A       9         8      0-UO       .18     .31 

B       12        38      0-150     -.05     .41 

of  this  sort.   It  should  be  made  clear,  however,  that  the  pro- 
posal teams  were  asked  to  include  only  that  literature  search 
conducted  specifically  to  assist  them  in  solving  the  problem. 
Literature  search  is  also  conducted  on  a  continuing  basis  to 
maintain  a  current  fund  of  knowledge  in  a  particular  discipline. 
It  is  conceivable  that  the  latter  would  show  a  higher  correla- 
tion with  technical  quality,  provided  it  could  be  measured 
properly, 

6 ,   Team  Size 

Team  size  is  defined  as  the  number  of  technical  employees 
on  the  proposal  team.   Although  it  might  be  postulated  that  a 
larger  team  would  include  a  greater  range  of  competences,  team 
size  apparently  bears  little  relation  to  technical  quality  on 
either  problem  (Table  VI). 
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TABLE  VI 
RELATION  OF  TEAM  SIZE  TO  TECHNICAL  QUALITY 

No.  of   Median   Range     Corre- 
Problem   labs    size   of  size   lation   p 

A       9        2      1-3       -.06   .46 

B       12         7      3-20       .08   .36 

7  0   Laboratory  Size 

Total  number  of  employees  at  the  local  establishment 

is  used  as  the  measure  of  laboratory  size.   In  general,  the 

larger  establishments  tend  to  allocate  more  effort  to  the 

proposal  preparation  (T72^=  '^^  for  Problem  A;  .58  for 

Problem  B)  and  to  submit  a  higher  bid  price. (T7g=  .35  for 

Problem  A;  .51  for  Problem  B) .   The  correlations  shown  in 

Table  VII  are  not  high  enough  to  justify  any  firm  conclusions 

about  the  relationship  to  technical  quality. 
TABLE  VII 

RELATION  OF  LABORATORY  SIZE  TO  TECHNICAL  QUALITY 

No.  of   Median   Range     Corre- 
Problem    labs     Size    of  Size   lation   p 

A        9        60     22-2000    .25    .20 

B      12      1600   115-15,000   .29    .09 

8.   Number  of  Technical  Personnel 


Technical  personnel  refers  to  the  number  of  engineers 
and  scientists  in  the  laboratory.   This  includes  both  graduate 


^The  symbol  T  stands  for  the  Kendall  tau  coefficient;  Ty2 
is  the  correlation  between  variable  7  and  variable  2  as  listed 
--"n  Table  XI. 
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engineers  and  scientists  and  nongraduate  engineers  who  are  in 
an  exempt  job  classification  (as  defined  by  the  Fair  Labor 
Standards  Act)„   One  might  expect  the  size  of  technical  work 
force  to  be  one  of  the  most  important  resources  for  proposal 
preparation,  and  indeed  it  is. 

TABLE  VIII 
RELATION  OF  NUMBER  OF  TECHNICAL  PERSONNEL  TO  TECHNICAL  QUALITY 

No.  of   Median   Range  of   Corre- 
Problem   labs   number   number   lation   p 

A       9        25      1-750      .39    .09 

B 12 550     10-2000     .41    .03 

Technical  quality  is  more  strongly  related  to  size  of  technical 
work  force  than  to  laboratory  size.   The  correlation  between 
laboratory  size  and  technical  quality  is  reduced  to  non-signifi- 
cance when  the  effect  of  size  of  technical  work  force  is  par- 
tialed  out,  (TyQ^g=  ,03  for  Problem  A;  -.10  for  Problem  B) , 

9 ,   Time  Spent  with  Laboratory  Specialists 
An  important  source  of  expertise  is  the  use  of  laboratory 
specialists  who  are  not  on  the  proposal  team  but  who  are 
available  to  them  for  consultation.   Table  IX  shows  that  this 
is  the  strongest  contributor  to  success  of  the  three  sources 
of  expertise  investigated. 
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TABLE  IX 

RELATION  OF  TIME  SPENT  WITH  LABORATORY  SPECIALISTS  TO  TECHNICAL 
QUALITY 

Noo  of   Median   Range  of   Corre- 
Problem    labs     Time     time lation   p 

A         9        1      0-20       ,37    ,13 

B 12 45 0-200 .55    .01 

10 .   Time  Spent  with  Outside  Consultants 
Outside  consultants  were  defined  to  the  respondents  as 
anyone  not  in  the  employ  of  the  laboratory  who  made  a  technical 
contribution  to  the  solution  on  any  basis,  paid  or  unpaid, 
formal  or  informal.   Most  of  the  consultation  was  obtained  on 
an  unpaid  informal  basis,  some  of  it  from  potential  vendors. 

TABLE  X 

RELATION  OF  TIME  SPENT  WITH  OUTSIDE  CONSULTANTS  TO  TECHNICAL 

QUALITY 


Problem 

No.  of 
labs 

Median 
Time 

Range  of 
time 

Corre- 
lation 

P 

A 
B 

9 
12 

0 
17,5 

0-25 
0-100 

-,11 
-.29 

.42 
,09 

The  use  of  outside  consultants  did  not  prove  to  be  of 
direct  benefit  in  solving  either  of  the  two  problems.   Time 
spent  with  outside  consultants  is  inversely  related  to  the 
percentage  of  technical  personnel  in  the  company  (T=  -.52  for 
Problem  A;  -.59  for  Problem  B.)   Percentage  of  technical 
personnel  may  be  considered  an  index  of  relative  orientation 
toward  research  or  manufacturing.  The  results  thus  would  indicate 
that  the  more  production-oriented  establishments  rely  more  heavily 
upon  outside  technical  consultation  in  preparing  R  S  D  proposals. 
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TABLE  XI 
SUMMARY  OF  INTERCORRELATIONS 


0  1           2  3  4  5  6  7  8  9  10 

0~.      Technical  TTT      731  7^  ^7^  TTS  TtTE  77!!  77^  TTl  vTT 
Quality 

1.       Cost                      .29  .03  t22  .06  .06  .31  .39  0  .49  .04 


2.  Level  of     ,33   .22        .45   .06   .21   .35   .40   .25  -rl2   .52 
Effort 

3.  Educa-       .10   .10   .54       -r04  -19   .30   .23   .15   -r52   .20 
tional 

Level 

4.  Time  in      .38   .18   .51   .19        .15  -20     0  -r38  -35   .49 
Analytic 

Design 

5.  Time  in     tO  5   .11   .15   .14   .38       "720   .15   .06   .19   .37 
Literature 

Search 

6.  Team  Size    .08   .42   .30   .11   .54   .31        .35   .31   .03   .17 


7.  LaLorn-      .29   .58   .42   .13   .29   rO 8   .51   .     .64   .28     0 
torv  Size    .......... 

8.  :!o.  of       .41   .45   .53   .32   .30   -08   .39   .82        .37   -40 
Technical 

Personnel 

9.  Time  with    .55   .09   .11   .06   .52  -r02   .27   .18   .18       -16 
Laboratory 

Specialists 
10.   Time  with   t29  -08  -rlO     0   .21   .42   .26   -r08   -14  v02 
Outside 
Consultants 

"   Intercorrelations  for  Problem  A  lie  above  the  diagonal,  those  for 
Problem  B  below.   For  Problem  A,  p  <0.10  when  T:^  .39,  p^0.05  when 
T>/.47,  for  Problem  B,  p<0.10  when  T>.  26,  p  <.0  5  v;hen  T::5,.37. 
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Number  of  Alternative  Approaches  Considered 
To  permit  further  analysis,  Problems  A  and  B  were 
divided  into  four  sub-problems  and  a  comparison  was  made  of 
the  problem-solving  processes  leading  to  a  laboratory's  two 
best  and  two  worst  solutions  among  the  sub-problems.   These 
comparisons  make  it  possible  to  assess  one  more  variable: 
the  number  of  alternative  approaches  considered  in  solving 
each  sub-problem.   When  tested  statistically,  this  proved 
to  be  a  significant  factor  (p=.02)  influencing  success  in 
the  ^solution  of  the  sub-problems.   More  extended  analysis  of 
the  role  of  prior  experience  and  of  problem-solving  tactics 
is  presented  in  another  report  (Allen  and  Marquis,  1963b). 

CONCLUSIONS 

In  two  proposal  competitions  for  government  R  S  D 
contract,  ten  characteristics  of  the  proposal  effort  have 
been  related  to  evaluated  technical  quality  of  the  proposal. 
Results  show  that: 

1.  Technical  quality  is  unrelated  to  the  size  of  the 
proposal  preparation  team, 

2.  Technical  quality  is  positively  related  to  the  size  of 
the  laboratory  and  especially  to  the  total  number  of  engineers 
and  scientists  in  the  laboratory. 

3.  Total  man-hours  of  technical  effort  in  proposal  prepar- 
ation is  positively  correlated  with  technical  quality. 

4.  Time  spent  consulting  specialists  in  the  laboratory  con- 
tributes definitely  to  technical  quality,  but  time  spent  con- 
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sulting' outside  the  laboratory  shows  no  (or  a  negative)  relation. 

5.  On  the  problem  calling  for  a  great  deal  of  analytic 
design  (Problem  B) ,  the  time  spent  in  analytic  design  was 
positively  related  (T=.38)  to  technical  quality.   On  Problem  A, 
which  required  little  analytic  design,  the  relation  was  nega- 
tive (T=t29). 

6 .  Time  spent  in  literature  search  was  unrelated  to  techni- 
cal quality. 

These  results  are  based  on  a  very  small  sample  of  research 
activities  and  should  be  interpreted  only  as  demonstrating  the 
feasibility  and  potential  usefulness  of  a  new  research  design 
for  the  comparative  study  of  factors  in  research  success. 
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